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Abstract 

There  is  increased  interest  in  the  use  of  innovative  modeling  and  simulation  (M&S)  tools  to  support  the 
acquisition  of  defense  systems.  This  paper  discusses  the  results  of  two  recent  activities  that  were  convened  to 
shed  light  on  this  issue:  a  study  by  the  National  Research  Council  (NRC)  entitled  "Modeling  and  Simulation 
in  Manufacturing  and  Defense  Systems  Acquisition:  Pathways  to  Success"  and  a  workshop  sponsored  by  the 
Office  of  the  Secretary  of  Defense  (OSD)  on  "M&S  to  Support  C4ISR  Acquisition  and  Transformation".  The 
paper  concludes  by  identifying  the  major  common  themes  that  emerged  from  the  two  activities  and  potential 
next  steps. 

A.  Introduction 

This  paper  identifies  and  discusses  recent  perspectives  that  have  emerged  on  the  use  of  modeling  and 
simulation  (M&S)  to  support  the  acquisition  of  defense  systems.  The  paper  begins  by  identifying  and 
analyzing  key  trends  that  are  likely  to  affect  the  acquisition  of  defense  systems  over  the  next  decade. 
Consistent  with  these  factors,  a  vision  for  a  reformed  acquisition  process,  simulation  based  acquisition 
(SBA),  is  introduced.  The  paper  then  derives  key  insights  on  SBA  that  have  emerged  from  several  recent 
events.  The  first  event  is  a  National  Research  Council  (NRC)  study  entitled  “Modeling  and  Simulation  in 
Manufacturing  and  Defense  Systems  Acquisition:  Pathways  to  Success".  The  basic  facts  of  the  study  are 
summarized  and  the  major  study  recommendations  are  presented  and  discussed.  That  is  followed  by  a 
description  of  the  second  event,  a  workshop  sponsored  by  the  Office  of  the  Secretary  of  Defense  (OSD)  on 
M&S  to  Support  C4ISR  Acquisition  and  Transformation.  That  event  is  characterized  by  the  insights  that 
were  developed  in  plenary  and  a  summary  of  the  major  recommendations  that  were  developed  by  the 
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workshop’s  break-out  groups.  The  paper  concludes  by  identifying  the  major  common  themes  that  emerged 
from  the  two  events  and  potential  next  steps. 

B.  Context 

As  context  for  this  paper,  this  section  briefly  analyzes  trends  affecting  Department  of  Defense  (DoD) 
acquisition  needs  and  formulates  a  new  acquisition  vision. 

B.l  Trends  Affecting  DoD  Acquisition  Needs 

There  are  several  major  trends  that  will  affect  the  M&S  and  ancillary  tools  that  are  needed  to  support  the 
acquisition  of  military  systems.  These  trends  are  depicted  in  Figure  1.  The  following  discussion  identifies 
these  trends  and  discusses  their  implications  on  the  use  of  M&S  in  the  acquisition  process. 


Figure  1.  Trends  Affecting  DoD  Acquisition  Needs 


B.1.1  International  Security  Environment 

The  dawn  of  the  21st  century  has  given  rise  to  several  fundamental  changes  in  the  international  security 
environment.  In  the  aftermath  of  September  11,  2001,  there  has  been  a  dramatic  increase  in  the  attention  paid 
to  homeland  security  issues.  Although  the  interest  in  potential  major  theater  wars  (MTWs)  has  also  persisted, 
the  recent  focus  has  been  on  states  that  have  access  to  chemical,  biological,  radiological,  nuclear,  and 
explosives  (CBRNE)  weapons.  There  is  particular  concern  that  these  actors  could  use  such  weapons 
asymmetrically  to  pose  anti-access  issues  that  would  create  challenges  for  conventional  forces.  In  addition, 
there  is  the  realization  that  operations  other  than  war  (OOTW)  (e.g.,  peacemaking,  peacekeeping)  are  likely 
to  recur,  prompting  the  need  to  create  ad  hoc  coalitions  of  willing  nation  states,  international  organizations, 
and  non-governmental  organizations. 

Currently,  there  are  few  existing  M&S  that  are  well  suited  to  address  these  issues.  Thus,  a  new  generation  of 
M&S  must  be  developed  for  these  conflict  environments  that  can  support  the  trade-off  studies  needed  to 
explore  meaningful  break-points  in  mission  capabilities. 
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B.1.2  Strategic  Vision 

Over  the  last  five  years,  a  series  of  strategic  products  has  emerged  to  guide  the  DoD's  responses  to  these 
changes  in  the  international  security  environment.  At  the  base  of  these  products  lies  Joint  Vision  2010  and 
2020  (Reference  1,  2)  issued  by  the  Chairman,  Joint  Chiefs  of  Staffs.  Those  visions  emphasized  that 
enhanced  command,  control,  communications,  computers,  intelligence,  surveillance,  and  reconnaissance 
(C4ISR)  will  be  the  critical  enabler  for  transforming  the  US's  military  force.  Building  on  that  vision,  the 
Secretary  of  Defense  in  his  Annual  Report  to  the  President  and  the  Congress  (ADR)  (Reference  3)  has 
identified  six  transformational  objectives  to  focus  the  actions  of  the  military.  Those  objectives  also  identify 
C4ISR  as  a  major  direct  and  indirect  factor  in  the  proposed  transformation. 

At  present,  there  are  relatively  few  M&S  that  credibly  reflect  C4ISR  processes  and  systems.  Thus,  efforts  are 
needed  to  deal  creatively  with  this  dimension  of  the  problem,  either  by  incorporating  the  effects  of  C4ISR  in 
existing  or  proposed  M&S,  or  by  interfacing  M&S  with  operational  C4ISR  systems.  In  addition,  those  latter 
experiences  could  be  used  to  collect  the  data  needed  to  improve  modeling  of  C4ISR. 

B.1.3  Resource  Trends 

Within  the  last  year,  there  has  been  a  renewed  commitment  by  the  US  to  devote  increasing  resources  to 
national  security.  This  increase  reflects  the  demands  from  multiple  needs:  enhanced  attention  to  homeland 
security,  as  well  as  the  need  to  sustain  selected  legacy  forces  while  pursuing  transformational  goals.  Even 
though  those  resources  are  substantial,  they  are  inadequate  to  deal  with  all  the  national  security  demands  for 
resources.  As  an  example,  the  Crusader,  a  highly  automated  artillery  system,  was  cancelled  to  free  up 
resources  for  more  pressing  systems.  Thus,  there  will  continue  to  be  strong  pressure  to  acquire  the  "right 
things"  as  well  as  acquiring  "things  right".  Consequently,  there  will  be  increasing  need  for  M&S  tools  that 
will  help  the  senior  leadership  understand  which  are  the  "right  things"  with  an  emphasis  on  proceeding  with 
only  those  ongoing  acquisitions  which  support  future  system-of-systems  concepts. 

In  order  to  reduce  the  military  footprint  in  the  theater  and  reduce  resource  requirements,  there  is  interest  in 
acquiring  systems  that  require  fewer  personnel  to  operate  them.  To  support  that  concept,  there  is  interest  in 
acquiring  M&S  that  support  much  more  effective  education  and  training,  any-where,  any-time. 

B.1.4  Institutional  Initiatives 

There  is  growing  concern  that  the  current  acquisition  process  is  stultifying,  limiting  the  Program  Manager's 
creativity  and  flexibility,  and  is  too  focused  on  the  acquisition  of  individual  systems  vice  military  capability. 
As  a  consequence,  a  revision  of  DoD  Directive  (DoDD)  5000  is  in  progress  that  is  briefer,  less  directive,  and 
incorporates  system-of-systems  acquisition.  Current  drafts  of  the  revised  documents  emphasize  processes 
such  as  evolutionary  acquisition  and  spiral  development.  Clearly,  there  is  a  need  for  enhanced  M&S  tools 
and  ancillary  data  to  help  implement  these  concepts,  particularly  to  address  the  complexities  inherent  in 
mission  capabilities  and  system-of-systems  acquisition  issues. 

B.1.5  Military  Systems 

Historically,  the  acquisition  of  systems  focused  on  individual  platforms,  keeping  most  key  operational  and 
support  factors  fixed  (e.g.,  concepts  of  operation,  education  and  training).  There  is  increased  understanding 
that  one  needs  to  acquire  and  assemble  systems-of-systems  to  realize  enhanced  mission  effectiveness.  This 
appreciation  has  led  to  increased  emphasis  on  interoperability  along  with  the  recognition  that  testbeds  or 
environments  are  needed  to  assess,  achieve,  and  sustain  it.  In  addition,  the  interest  in  transforming  the 
military  has  stimulated  the  re-thinking  and  re-orienting  of  all  aspects  of  doctrine,  organization,  training, 
materiel,  leadership  and  education,  personnel,  and  facilities  (DOTML-PF).  Currently,  existing  tools  lack  the 
flexibility  to  vary  all  of  these  factors  simultaneously.  Thus,  attention  is  being  given  to  new  M&S  techniques 
(e.g.,  agent  based  modeling)  which  have  the  potential  to  deal  flexibly  with  a  broader  set  of  these  factors  and 
their  interrelationships. 
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B.1.6  Commercial  Technology 

In  recent  years,  there  has  been  a  dramatic  increase  in  the  use  of  commercial-off-the-shelf  (COTS)  technology 
in  military  systems,  particularly  for  C4ISR  systems.  These  COTS  products  tend  to  be  highly  dynamic, 
spawning  upgraded  versions  on  the  order  of  6  to  18  months.  Since  this  is  much  shorter  than  the  historical 
DoD  acquisition  cycle,  it  is  stimulating  the  use  of  innovative  acquisition  strategies  (e.g.,  evolutionary 
acquisition)  to  avoid  fielding  products  that  are  technologically  obsolete.  To  support  this  process,  there  is  a 
need  for  M&S  testbeds  to  explore  the  value  of  injecting  new  technology  on  mission  effectiveness,  in  a  timely 
manner. 

B. 2  Acquisition  Vision 

To  respond  to  these  major  trends,  a  vision  of  a  new  acquisition  paradigm  is  emerging  that  yields  substantial 
reductions  in  time,  resources,  risk,  and  total  ownership  costs  throughout  the  life  cycle  process,  while 
simultaneously  increasing  the  system’s  quality,  military  worth,  and  supportability. 

In  order  to  achieve  those  benefits,  it  is  perceived  that  the  intelligent  use  of  simulations  is  the  critical  enabler. 
These  simulations  must  be  robust,  used  collaboratively  by  all  of  the  stakeholders  involved  in  the  acquisition, 
and  integrated  across  the  phases  and  functions  of  the  system  life  cycle.  In  addition,  to  take  full  advantage  of 
the  investments  in  these  simulations,  steps  should  be  taken  to  ensure  that  they  are  reused  to  support  other, 
related  system  programs.  This  philosophy  of  employing  M&S  extensively  and  consistently  within  and  across 
program  lines  is  often  referred  to  as  Simulation  Based  Acquisition  (SBA). 

C.  Key  Insights 

Over  the  last  several  years,  several  activities  have  explored  options  to  enhance  the  use  of  M&S  to  support 
system  acquisition.  This  section  characterizes  the  insights  that  have  emerged  from  two  of  those  activities:  a 
NRC  study  and  an  OSD-sponsored  workshop. 

C.l  National  Research  Council  Study  Results 

In  2001,  the  Defense  Modeling  &  Simulation  Office  (DMSO)  requested  that  the  NRC  investigate  next- 
generation  evolutionary  and  revolutionary  M&S  capabilities  that  will  support  enhanced  defense  systems 
acquisition.  The  NRC  is  the  principal  operating  agency  of  both  the  National  Academy  of  Sciences  and  the 
National  Academy  of  Engineering  in  providing  analytical  services  to  the  government,  the  public,  and  the 
scientific  and  engineering  communities.  The  study  was  chaired  by  Peter  Castro,  Eastman  Kodak  Corporation, 
and  performed  by  a  committee  of  twelve  experts  on  M&S,  manufacturing,  and  acquisition,  drawn  from 
academia  (i.e.,  Harvard,  California  Institute  of  Technology,  University  of  Arizona,  Carnegie  Mellon 
University,  Old  Dominion  University,  Johns  Hopkins  University/ Applied  Physics  Lab,  and  the  University  of 
Wisconsin-Milwaukee),  industry  (i.e.,  Lockheed  Martin,  Ford,  GRCI),  and  Federally  Funded  Research  & 
Development  Centers  (FFRDCs)  (i.e.,  MITRE,  and  Sandia  National  Labs,).  This  group  received  extensive 
briefings  from,  and  conducted  in-depth  discussions  with,  innovators  in  SBA  in  government  and  industry.  The 
final  report  was  issued  in  the  summer  of  2002  (Reference  4). 

The  NRC  Panel  proposed  four,  broad,  inter-related  recommendations.  As  a  key  enabler,  it  is  vital  that  the 
community  invest  in  appropriate  technology  and  research.  The  results  of  these  efforts  will  give  rise  to  a 
community- wide  infrastructure  that  supports  increased  consistency  and  integration.  Use  of  that  infrastructure 
will  provide  experience  that  will  guide  further  use  as  well  as  pointing  to  important  opportunities  for  further 
research.  Finally,  people  and  culture  are  the  key  factors:  if  the  people  and  the  culture  in  which  they  operate 
do  not  trust  and  embrace  the  creative  application  of  M&S  to  acquisition,  SBA  will  never  become  a  wide¬ 
spread  activity. 
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The  Panel  characterized  this  set  of  recommendations  as  a  “virtuous 


auto-catalytic  cycle”.  That  meant  that 
is 

,  .  .  .t1  i  J  i  ^  the  others.  Thus,  once  a  critical  mass  is 

each  of  the  recommendations  will  serve  to  stimulate  and  reinforce 

achieved,  the  activities  will  be  self-sustaining.  However,  it  underscores 

more  of  these  recommendations  could  undermine  the  entire  enterprise. 


the  fact  that  failure  to  perform  one  or 


C.1.1  Technology  and  Research  Recommendations 

In  the  broad  area  of  technology  and  research ,  the  Panel  recommended  that  “Long  term  R&D  should  be 
funded,  conducted,  and  applied  to  enhance  the  science  and  technology  base  for  M&S  in  ...  manufacturing, 
acquisition,  and  life-cycle  support  of  military  systems.”  To  amplify  on  that  overarching  recommendation,  the 
Panel  formulated  four  subordinate  recommendations  for  DoD: 

•  Conduct  or  support  basic  research  and  development  in  the  following  areas:  modeling  methods 
(e.g.,  scalability,  multi-resolution  and  multi-viewpoint  modeling,  agent-based  modeling);  model 
integration  (e.g.,  interoperability,  composability);  model  correctness  (e.g.,  verification,  validation, 
and  accreditation  (VV&A));  standards  (e.g.,  M&S  standards  for  interoperability  and  modeling); 
methods  and  tools  (e.g.,  for  assistance  in  the  translation  of  system  requirements  into  system 
functionality);  and  domain-specific  models  (e.g.,  M&S  for  operations  other  than  war). 

•  Enhance  the  ability  to  deal  with  systems-of-sy stems.  This  should  include  the  generation  of  a  library 
of  composable  system  models,  the  ability  to  manage  interactions  among  component  systems 
efficiently,  and  the  development  of  efficient  experimental  design  techniques. 

•  Create  a  research  initiative  at  multiple  universities.  This  initiative  should  focus  on  the  key  M&S 
shortfalls  identified  by  DoD  program  offices. 

•  Plan  and  execute  the  transition  of  research  into  applications  as  an  integral  part  of  the  development 
process. 

C.1.2  Infrastructure  Recommendations 

In  the  area  of  infrastructure  for  M&S,  the  Panel  recommended  that  the  community  should  “...Invest  in 
‘common  good’  activities  to  encourage  standards  and  strong  infrastructure  for  M&S”.  To  clarify  that 
recommendation,  the  panel  formulated  three  subordinate  recommendations  for  DoD: 

•  Institute  incentives  for  program  managers  to  develop  M&S  elements  that  contribute  to  the  general 
infrastructure.  This  should  include  an  annual  competition  for  the  best  infrastructure  contributions. 

•  Exploit  common  elements  of  M&S  to  develop  a  common  infrastructure  capable  of  supporting 
consistency  and  interoperability  across  programs.  This  infrastructure  should  include  common 
repositories  that  can  support  multiple  phases  of  a  program  as  well  as  multiple  programs,  a  trained 
M&S  workforce,  and  an  information  technology  infrastructure  that  will  drive  the  advance  of  the 
needed  M&S  infrastructure. 

•  Advance  the  emergence  of  common  standards  for  performance  simulation  and  product  modeling. 
To  that  end,  DoD  should  remain  actively  engaged  in  commercial  standards  efforts,  take  the  lead  in 
the  development  of  standards  that  lack  commercial  interest,  and  develop  standard  semantics  for  the 
data  elements  used  in  DoD  acquisition  related  M&S. 


C.1.3  Use  of  M&S  Recommendations 

In  the  broad  area  of  the  use  of  M&S  in  acquisition  and  manufacturing,  the  Panel  recommended  that  “Process 
improvements  should  be  undertaken  to  better  support  integration  of  M&S  within  DoD’s  system  acquisition 
process.”  To  expand  on  that  overarching  recommendation,  the  Panel  formulated  four  subordinate 
recommendations  for  DoD: 

•  Expand  M&S  in  the  concept  exploration  phase ,  to  ensure  that  we  “build  the  right  thing”  as  well  as 
“building  the  thing  right”.  To  that  end,  M&S  should  be  used  more  extensively  during  the 
requirements  process. 

•  Develop  a  set  of  guidelines  and  best  practices  for  ownership  rights  among  DoD  and  industry  (with 
respect  to  M&S  and  data)  to  facilitate  the  potential  for  collaboration  and  the  reuse  of  M&S. 


RTO-M  P-094 


17-5 


Perspectives  on  the  Use  of  M&S  to  Support  Systems  Acquisition 


ORGANIZATION 


•  Define  how  M&S  should  be  integrated  into  the  DoD  system  acquisition  process.  This  should 
include  the  use  of  the  maturity  of  the  simulation  support  plan  as  an  element  in  milestone  decision 
reviews. 

•  Create  and  implement  incentives  for  DoD  Program  Managers  (PMs)  to  adopt  best  practices  on 
M&S  use. 

•  Define  and  undertake  pilot  efforts  at  the  OSD  level.  These  pilot  efforts  should  be  designed  to 
explore  cross-program  benefits  of  M&S  and  system-of-systems  issues. 

C.1.4  Culture  and  Human  Issues  Recommendations 

In  the  area  of  culture  and  human  issues ,  the  Panel  recommended  that  “...DoD  must  provide  leadership  to 
initiate,  support,  and  sustain  a  cultural  change  in  the  acquisition  process.”  To  amplify  on  that  overarching 
recommendation  the  Panel  formulated  three  subordinate  recommendations  for  DoD: 

•  Fundamentally  transform  the  current  acquisition  culture  in  DoD  into  one  characterized  by 
collaboration,  cumulative  learning,  agility,  risk  tolerance,  learning  from  failure,  and  appropriate 
rewards  and  penalties. 

•  DoD  should  take  the  lead  in  collaborating  with  academia  and  industry  to  build  the  intellectual 
capital  needed  to  implement  SBA.  This  should  include  the  support  of  existing  and  developing 
academic  degree  programs  in  M&S,  the  establishment  of  a  mentoring  program,  and  the 
encouragement  of  individuals  to  view  M&S  as  a  "lifetime  learning"  endeavor. 

•  DoD  should  establish  a  center  of  excellence  for  M&S  in  SBA.  This  resource  would  help  create  and 
promulgate  the  desired  acquisition  culture  and  enhance  DoD's  ties  to  the  academic  community. 

C.2  OSD  Workshop  Results 

The  purpose  of  the  OSD  Workshop  was  to  identify  high  priority  M&S  needs  and  recommend  associated 
policy  changes  and  initiatives  to  support  the  acquisition  and  transformation  of  C4ISR  systems.  Note  that  by 
focusing  on  C4ISR  systems,  this  activity  was  more  restricted  in  scope  than  the  NRC  study  that  addressed  the 
acquisition  of  all  DoD  systems.  However,  during  the  course  of  the  workshop  deliberations,  there  was  a  sense 
that  many  of  the  issues  faced  in  the  acquisition  of  C4ISR  systems  map  to  those  of  complex  battle 
management  systems  that  integrate  C4ISR  and  weapons  systems.  It  is  envisioned  that  the  results  will  provide 
input  to  a  subsequent  investment  plan  for  M&S  to  support  the  acquisition  and  transformation  of  C4ISR 
systems. 

The  Workshop  was  sponsored  by  three  organizations  in  the  OSD.  They  include  the  Assistant  Secretary  of 
Defense  (C3I),  and  two  offices  in  the  Undersecretary  of  Defense  (Acquisition,  Technology,  and  Logistics): 
the  Interoperability  Office  and  the  Defense  Modeling  and  Simulation  Office.  To  ensure  a  strong  industry 
component  to  the  workshop,  the  AIAA  Technical  Committee  on  Information  and  C2  Systems  worked 
cooperatively  with  the  workshop  planning  group  to  provide  knowledgeable  participants  across  the  areas  and 
in  the  synthesis  activity. 

The  heart  of  any  workshop  is  the  experts  that  are  assembled  to  deliberate  on  the  issues  of  interest.  In  this 
case,  forty-four  experts  on  acquisition,  C4ISR,  and  modeling  and  simulation  were  brought  together.  These 
experts  provided  a  balanced  mix  of  representatives  from  government,  industry,  and  FFRDCs. 

In  order  to  achieve  the  goals  and  objectives  of  the  workshops,  a  series  of  plenary  and  break-out  sessions  were 
held.  In  plenary,  the  sponsors  proved  individual  charges  to  the  participants.  That  was  followed  by 
presentations  on  institutional  perspectives  with  emphasis  on  the  DoDD  5000,  governing  the  acquisition  of 
systems.  The  plenary  session  concluded  with  Service  presentations  on  innovative  activities  to  enhance  the 
acquisition  of  complex  systems. 

Subsequently,  the  assembled  experts  were  divided  into  break-out  groups  by  system  type  (i.e., 
communications,  sensors,  C2  systems/information  processing)  and  systems-of-systems.  Those  break-out 
groups  began  their  deliberations  with  a  list  of  strawman  M&S  capability  objectives.  They  evaluated 
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associated  M&S  needs,  and  identified  the  highest  priority  M&S  shortfalls.  They  subsequently  identified 
options  to  ameliorate  those  highest  priority  shortfalls.  The  Workshop  was  held  2-4  April  at  the  MITRE 
Corporation,  McLean,  VA  (Reference  5). 

C.2.1  Plenary  Insights 

The  plenary  session  began  with  presentations  by  the  workshop  sponsors  or  their  representatives.  All  see 
M&S  as  critical  to  their  organization’s  mission  effectiveness.  With  respect  to  systems-of-systems,  all  saw  it 
as  a  wave  of  the  future,  which  poses  new  M&S  challenges  for  the  community. 

The  presentation  on  DoD  acquisition  directives  began  by  citing  challenges  in  overcoming  the  shortfalls  in  the 
existing  acquisition  process  (e.g.,  it  takes  too  long,  costs  too  much,  and  is  incompatible  with  modern 
subsystem  cycle  times).  The  opportunities  offered  by  the  acquisition  model  described  in  the  current  version 
of  DoDD  5000  were  cited.  It  was  stressed  that  the  model  emphasizes  the  use  of  evolutionary  acquisition, 
anticipating  the  need  to  inject  new  technology  periodically  (e.g.,  on  the  order  of  every  eighteen  months  to 
keep  pace  with  developments  in  the  information  systems  sector  of  the  commercial  world).  Several  enhanced 
roles  for  M&S  in  support  of  this  new  model  were  identified.  These  included  support  to  Analyses  of 
Alternatives  (AoAs)  in  Concept  &  Technology  Development  (CDT),  Early  Operational  Assessments  (EOAs) 
in  System  Development  &  Demonstration  (SDD),  and  Operational  Test  &  Evaluation  (OT&E)  in  Production 
&  Deployment  (P&D). 

However,  it  was  acknowledged  that  there  are  several  major  residual  challenges  in  DoDD  5000  that  require 
future  action,  particularly  in  the  guidance  it  provides  on  system-of-systems  acquisition.  For  example,  since 
individual  systems  are  acquired  asynchronously,  how  do  you  do  “full  system  demonstration”  before  the 
commitment  to  production  or  perform  T&E  of  a  full  system-of-systems?  In  addition  how  do  you  deal  with 
interoperability  and  supportability  of  a  C4ISR  system  when  multiple  configurations  are  deployed, 
simultaneously?  This  latter  issue  is  of  particular  concern  to  the  US  Army  that  must  deal  with  the 
simultaneous  fielding  of  large  numbers  of  legacy,  interim,  and  objective  systems.  As  noted  in  Section  B.1.4 
of  this  paper,  since  the  workshop  was  completed  there  has  been  renewed  interest  in  revising  the  DoDD  5000 
series  to  enable  an  acquisition  policy  environment  that  fosters  efficiency,  creativity,  and  innovation.  Efforts 
to  achieve  those  objectives  are  currently  underway. 

In  the  final  segment  of  the  plenary,  representatives  from  the  three  Services  described  key  initiatives.  The 
Electronic  Systems  Command,  US  Air  Force,  is  developing  a  Joint  Synthetic  Battlespace  (JSB)  to  support 
the  future  acquisition  and  integration  of  C4ISR-weapon  systems-of-systems.  This  effort  has  pursued  a  “top 
down”  approach,  systematically  addressing  the  dimensions  of  leadership,  policy,  process,  technology,  and 
resources.  To  take  advantage  of  the  extensive  investment  in  M&S  environments  that  have  been  made,  this 
approach  strongly  emphasizes  Service/industry  collaboration. 

Second,  the  US  Navy  is  developing  a  Naval  Collaborative  Engineering  Environment  to  support  Program 
Executive  Officers  (PEOs)  and  PMs  in  meeting  the  Service’s  integration  and  interoperability  requirements. 
They  have  adopted  a  framework  for  integrated  system  evolution  that  emphasizes  the  separation  of  data  and  a 
suite  of  multi-functional  tools  to  support  the  initial  phase  of  acquisition  (e.g.,  mission  capability  packages 
supported  by  Operational  Requirement  Documents,  Analyses  of  Alternatives). 

Finally,  the  US  Army  completed  the  Service  perspectives  by  describing  current  thoughts  on  the  C4ISR- 
weapon  system  mix  for  the  proposed  Future  Combat  System  (FCS)  and  the  tools  that  are  evolving  to  support 
the  acquisition  of  that  capability.  In  the  latter  area,  they  are  creating  an  arena  to  evaluate  future  system 
concepts  drawing  on  the  capabilities  developed  at  the  Joint  Precision  Strike  Demonstration  (JPSD)  and  the 
Joint  Virtual  Battlespace  (JVB).  The  philosophy  is  to  integrate  existing  M&S  tools  and  resources  from  a 
variety  of  sources  (e.g.,  US  Army  Research,  Development,  and  Engineering  Centers  (RDECs),  National 
Labs)  and  to  augment  them  via  a  spiral  build  to  address  specific  system  acquisitions. 
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Looking  across  the  presentations  by  the  three  Services,  it  is  noteworthy  that  cross-Service  collaboration  on 
their  activities  is  ongoing  and  increasing.  Since  each  Service  has  employed  a  different,  complementary, 
approach  to  the  issue,  such  collaboration  has  the  potential  to  enhance  the  quality  eacb  Service  s 
environments  and  to  minimize  duplication  of  efforts.  From  a  system-of-systems  standpoint,  integrating 
across  the  three  Services  and  with  coalition  partners  remains  a  key  challenge. 


C.2.2  Break-out  Group  Insights 

Figure  2  provides  a  framework  that  identifies  a  set  of  M&S-related  categories  that  must  be  addressed  by  the 
acquisition  community.  The  break-out  groups  concluded  that  if  future  C4ISR  assessments  are  to  be  planned 
and  conducted  successfully  in  a  dynamic  environment  and  supported  effectively  by  the  M&S  community,  we 
must  consistently  address  all  of  these  factors  in  a  balanced  way.  The  backdrop  for  these  factors  is  set  by  the 
cultures  of  the  many  communities  that  must  participate  in  future  C4ISR  acquisitions.  To  remind  us  of  the 
importance  of  the  cultural  dimension,  one  facet  of  the  pyramid  is  labeled  “change  in  attitude,  culture”. 


The  base  of  the  framework  rests  on  support  tools  and  foundation  data  and  information.  It  subsumes 
repositories  of  critically  needed  information  (e.g.,  environmental  data).  Building  on  the  foundation 
data/information  is  a  set  of  broadly  needed  support  tools.  These  include  readily  tailorable  and  reusable 
scenarios.  Resting  on  these  support  tools,  are  sets  of  hierarchies  of  measures  of  merit  (MoMs).  These  range 
from  measures  of  C4ISR  system  performance  (e.g.,  bandwidth  of  communications  systems)  to  overall 
measures  of  campaign  effectiveness  (e.g.,  the  time  required  to  halt  an  invading  army).  Once  the  MoMs  have 
been  identified,  it  serves  to  clarify  the  individual  M&S  modules  that  are  needed  to  evaluate  the  measures  of 
interest.  These  M&S  modules  are  then  federated  into  environments  that  enable  the  acquirer  to  assess  the 
measures  efficiently  over  the  requisite  set  of  scenarios  and  assumptions.  In  order  to  develop  and  employ 
these  environments,  it  is  necessary  to  have  sound  policy,  management,  and  associated  processes  to  guide  the 
acquirer  in  applying  M&S  appropriately  in  the  acquisition  process. 


In  the  area  of  policy,  management  and  processes,  several  broad  needs  were  identified.  First,  the  system-of- 
systems  panel  called  for  a  revisit  of  DoDD  5000  to  ensure  that  it  addressed  critical  system-of-systems  issues 
adequately.  For  example,  there  was  a  need  to  address  mission  capabilities  based  decision-making  vice 
individual  system  decision  making.  Second,  there  is  a  need  to  develop  and  provide  government/contractor 
access  to  authoritative  M&S  and  repositories  for  data,  algorithms,  joint  scenarios,  and  synthetic  natural 
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environments.  Finally,  the  system-of-systems  panel  called  out  the  need  to  derive  insights  from  system-of- 
systems  events  (e.g.,  exercises,  experiments).  In  order  to  do  so,  there  is  a  need  for  sufficient  funding, 
authority,  and  responsibility  to  capture  and  exploit  important  data  from  these  events. 


Each  of  the  panels  identified  major  needs  in  the  area  of  acquisition  environments.  The  broadest  need  was 
articulated  by  the  system-of-systems  panel  that  called  out  the  need  for  secure,  distributed,  scalable, 
responsive,  standards-based,  collaborative  engineering  environments.  Specific  attributes  of  these 
environments  were  identified  by  the  other  panels.  For  example,  the  C2  panel  called  out  the  need  for  these 
environments  to  be  interoperable  with  those  of  industry  and  to  evolve  throughout  the  acquisition  process. 
The  remaining  panels  noted  the  need  for  these  environments  to  be  able  to  interface  with  other  systems,  M&S, 
humans  in  the  loop,  and  hardware  in  the  loop. 

Each  of  the  panels  identified  major  needs  in  the  area  of  M&S.  At  a  macro-level,  the  system-of-systems  panel 
called  out  the  need  for  a  reference  model  for  alternative  levels  of  interest  (e.g.,  system,  function,  mission, 
campaign)  that  would  help  identify  key  M&S  capabilities  and  shortfalls.  Several  of  the  other  panels 
identified  specific  M&S  that  were  needed.  These  included  communications  models  appropriate  for  system- 
of-system  level  analyses  and  network  operations  support,  planning,  and  training;  high  level  M&S  that  could 
support  quick  turn-around  assessments  for  sensor  trade  studies;  the  ability  to  model  information 
infrastructures;  and  M&S  that  featured  better  representations  of  decision  making  processes  in  C2. 

There  was  general  agreement  that  a  hierarchy  of  MoMs  is  needed  that  would  support  assessments  of  the 
impact  of  C4ISR  systems  on  mission  effectiveness.  This  becomes  particularly  important  in  addressing 
system-of-systems  issues,  where  MoMs  are  needed  which  reflect  the  performance  of  the  system-of-systems 
capabilities  as  well  as  the  contribution  of  individual  systems,  In  the  discussion,  it  was  noted  that  NATO’s 
evolving  Code  of  Best  Practice  for  C2  Assessment  (Reference  6)  is  an  excellent  point  of  departure  to  pursue 
the  development  of  those  MoMs.  These  analytic  constructs  are  critical  if  M&S  is  to  be  an  effective  tool. 

Several  panels  identified  the  need  for  support  tools  to  enhance  the  ease  of  use  and  responsiveness  of  M&S.  In 
particular,  it  was  stated  that  there  is  a  need  for  reusable,  tailorable  scenarios;  a  joint  library  with  an  extensive 
scenario  set  (including  Blue  force  laydowns);  and  common  environmental  representations. 


Finally,  several  of  the  panels  identified  the  need  for  key  foundation  data  and  information.  From  a  system-of- 
systems  perspective,  it  was  observed  that  there  is  a  need  for  synchronization  points  for  past,  current,  and 
future  system  performance  data.  In  addition,  needs  were  identified  for  common  standards  for  inputs  to  drive 
models  and  the  identification  of  architectural  data  to  link  architectural  representations  to  executable 
simulations. 

D.  Summary 

In  order  to  support  the  acquisition  and  transformation  of  C4ISR  systems,  a  clear  “bottom  line”  emerged  from 
these  recent  initiatives.  First,  both  events  emphasized  that  it  is  essential  that  a  cultural  transformation  be 
undertaken  in  the  acquisition  enterprise.  This  includes  strong  institutional  leadership,  the  implementation  of 
appropriate  incentives,  and  the  institutionalization  of  an  effective  education  and  training  effort.  Second,  it 
was  concluded  that  a  system-of-systems  perspective  is  vital.  It  is  no  longer  viable  to  restrict  acquisition  to  a 
single  system,  in  isolation.  Third,  M&S  was  seen  to  be  a  critical  enabler  of  the  acquisition  process.  In 
particular,  a  strong,  integrated  M&S  capability  is  required,  in  two  dimensions.  The  M&S  capability  must 
support  the  full  life  cycle  of  an  individual  acquisition  (i.e.,  intra-program)  as  well  as  cross-program 
acquisition  (i.e.,  inter-program).  Finally,  in  order  to  meet  the  M&S  needs  of  the  acquisition  community,  a 
balanced  set  of  M&S  initiatives  must  be  pursued.  These  range  from  initiatives  on  creating  and  making 
available  foundation  data/information  through  the  formulation  of  revised  high  level  policy  governing  system- 
of-systems  acquisition. 


As  a  consequence  of  these  events,  preliminary  plans  for  M&S  investments  to  support  C4ISR  acquisitions  are 
emerging.  There  is  interest  in  pursuing  a  range  of  initiatives  including  a  study  of  the  tools  needed  to  assess 
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proposed  operational,  system,  and  technical  architectural  products,  an  assessment  of  existing  and  emerging 
M&S  applications  to  support  SBA,  an  assessment  of  the  state-of-the-practice  in  integrated  M&S 
environments,  and,  ultimately,  an  application  of  these  tools  and  techniques  to  the  evolving  operational  and 
systems  concepts  for  network  centric  warfare. 
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International  Security 

■ 

Strategic  Vision 

Environment 

•  Homeland  Security 

- > 

•  Joint  Vision  2010,  2020 

•  MTWs 

•  Military  transformation 

•  Asymmetric  threats 

•  ADR  2002 

•  OOTWs 

H 

DoD 

Acquisition 

Needs 


Military  Systems 
System-of-Systems 


Acquisition  Vision 


•  What 

-  Reduce:  time,  resources,  risk,  and  total 
ownership  costs  throughout  life  cycle  process 

-  Increase:  quality,  military  worth,  and 
supportability  of  fielded  systems 

•  How:  Enabled  by  modeling,  simulation, 

and  information  technology  that  is 

-  Robust 

-  Used  collaboratively 

-  Integrated  across  life  cycle  phases,  functions, 
and  programs 


Simulation  Based  Acquisition 
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Essential  NRC  Study  Facts 

•  Why:  Investigate  next-generation  evolutionary  and 
revolutionary  M&S  capabilities  that  will  support 
enhanced  defense  systems  acquisition 

•  Who: 

-  Study  requested  by  Defense  Modeling  &  Simulation 
Office  (DMSO) 

-  Study  chaired  by  Peter  Castro,  Eastman  Kodak 
Corporation 

-  Twelve  panelists  from  academia,  industry,  FFRDCs 

•  How:  Extensive  briefings,  discussions  with 
government  and  industry  innovators 

•  When:  Report  issued  Summer  2002 
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Major  NRC  Study 
Recommendations  (1  of  2) 

Technology  &  Research:  “Long  term  R&D  should  be  funded, 
conducted,  and  applied  to  enhance  the  science  and  technology 
base  for  M&S  in  ...  manufacturing,  acquisition,  and  life-cycle 
support  of  military  systems” 

-  Conduct  basic  research  &  development 

-  Enhance  the  ability  to  deal  with  systems-of -systems 

-  Create  a  research  initiative  at  multiple  universities 

-  Plan  and  execute  the  transition  of  research  into  applications 

Infrastructure  for  M&S: "...  Invest  in  “common  good”  activities  to 
encourage  adequate  standards  and  a  strong  infrastructure  for  M&S” 

-  Institute  incentives  for  program  managers  to  develop  common  M&S 
elements 

-  Develop  a  common  infrastructure  capable  of  supporting  cross-program 
consistency,  interoperability 

-  Advance  common  standards 


Major  NRC  Study 
Recommendations  (2  of  2) 

Use  of  M&S  in  Acquisition  and  Manufacturing:  “Process 
improvements  should  be  undertaken  to  better  support  integration  of 
M&S  within  DoD’s  system  acquisition  process” 

-  Expand  M&S  in  the  concept  exploration  phase 

-  Develop  a  set  of  guidelines  and  best  practices  for  ownership  rights 

-  Define  how  M&S  should  be  integrated  into  the  DoD  system  acquisition 
process 

-  Create  and  implement  incentives  for  DoD  PMs  on  M&S  use 

-  Define  and  undertake  pilot  efforts  at  the  OSD  level 

Culture  and  Human  Issues: "...  DoD  must  provide  leadership  to 
initiate,  support,  and  sustain  a  cultural  change  in  the  acquisition 
process” 

-  Fundamentally  transform  the  current  acquisition  culture  in  DoD 

-  Build  intellectual  capital  needed  to  implement  SBA 

-  Establish  a  center  of  excellence  for  M&S  in  SBA 


Essential  OSD  Workshop  Facts 

•  Why:  Provide  input  to  an  investment  plan  for  M&S  to 
support  the  acquisition  and  transformation  of  C4ISR 
systems 

•  Who: 

-  Three  sponsors:  ASD(C3I);  OUSD(AT&L)  {10,  DMSO} 

-  Participants:  44  experts  from  government,  industry,  FFRDCs 

•  How: 

-  Plenary:  Perspectives  of  sponsors,  OSD,  Services 

-  Break-out  groups:  Organized  by  System  type 
(Communications,  Sensors,  C2/lnformation  Processing); 
Systems-of-Systems 

•  Where  &  When:  The  MITRE  Corporation,  McLean,  VA, 

2  -  4  April  2002 


17-9 


Plenary  Insights  (1  of  2) 


Sponsors’  perspectives:  Systems-of-systems 

are  perceived  as 

-  The  wave  of  the  future 

-  A  major  challenge  for  the  M&S  community 

Institutional  perspectives 

-  Acquisition  process  is  beset  with  major  problems 
(e.g.,  takes  too  long;  costs  too  much;  incompatible 
with  modern  subsystem  cycle  times) 

-  Although  the  DoDD  5000  model  emphasizes 
Evolutionary  Acquisition  it  does  not  address 
system-of-system  issues  adequately 


Plenary  Insights  (2  of  2) 

Service  Perspectives 

-  All  are  developing  M&S  environments  to  support  the 
acquisition  of  Systems-of-Systems;  e.g., 

•  US  Air  Force:  Joint  Synthetic  Battlespace  (JSB) 

•  US  Navy:  Collaborative  Engineering  Environment  (CEE) 

•  US  Army:  Joint  Virtual  Battlespace  (JVB) 

-  There  are  interesting  opportunities  to  collaborate 
and  coordinate  efforts 


Framework 
Recommend 


a 


or 

[ions 
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Summary  of  Major 
Recommendations  (1  of  2) 

*  Policy  and  Management/Processes: 

-Revise  DoDD  5000  to 

•  Clarify  the  acquisition  of  a  system-of-systems 

•  Focus  acquisition  process  on  achieving  mission 
capability 

-Provide  access  to  M&S,  data 
-Derive  insights  from  system-of-systems  events 


*  Environments  and  M&S  products: 

-  Create  shared  collaborative  environments  in 
which  to  address  interoperability  issues 

-  Develop  a  reference  model  of  cross¬ 
functional,  cross-hierarchy  M&S 


Policy 


Environment 
M&s  Products 
MoMs 

Support  Tools 


Summary  of  Major 
Recommendations  (2  of  2) 


•  MoMs: 


-Develop  sets  of  measures  of  system 
performance  linked  to  measures  of  mission 
effectiveness 


*  Support  Tools  and  Foundation 
Data/Information: 


-Create  common  environmental 
representations  and  libraries  of  reusable 
scenarios 

-Create  easily  accessible,  broadly  available 
data  on  architectures,  systems  performance, 
organizational  behavior 


17-14 


Summary 


In  order  to  support  the  acquisition  of  future  DoD 
systems 

-  A  cultural  transformation  is  essential 

-  A  system-of-systems  perspective  is  vital 

-  A  strong,  integrated  M&S  capability  is  required 

-  A  balanced  set  of  M&S  initiatives  (e.g.,  from  policy 
through  foundation  data/information)  must  be 
pursued  to  achieve  the  needed  M&S  capability 


